Virtual-reality-based digital twin of office spaces with social distance measurement feature

Background: Social distancing is an effective way to reduce the spread of the SARS-CoV-2 virus. Many students and researchers have already attempted to use computer vision technology to automatically detect human beings in the field of view of a camera and help enforce social distancing. However, because of the present lockdown measures in several countries, the validation of computer vision systems using large-scale datasets is a challenge. Methods: In this paper, a new method is proposed for generating customized datasets and validating deep-learning-based computer vision models using virtual reality (VR) technology. Using VR, we modeled a digital twin (DT) of an existing office space and used it to create a dataset of individuals in different postures, dresses, and locations. To test the proposed solution, we implemented a convolutional neural network (CNN) model for detecting people in a limited-sized dataset of real humans and a simulated dataset of humanoid figures. Results: We detected the number of persons in both the real and synthetic datasets with more than 90% accuracy, and the actual and measured distances were significantly correlated (r=0.99). Finally, we used intermittent-layer- and heatmap-based data visualization techniques to explain the failure modes of a CNN. Conclusions: A new application of DTs is proposed to enhance workplace safety by measuring the social distance between individuals. The use of our proposed pipeline along with a DT of the shared space for visualizing both environmental and human behavior aspects preserves the privacy of individuals and improves the latency of such monitoring systems because only the extracted information is streamed. © 2021 Beijing Zhongke Journal Publishing Co. Ltd

Generating Synthetic Data for Deep Learning using VR Digital Twin

The ongoing Covid-19 pandemic has made it challenging for large scale data collection, in particular for Convolutional Neural Network (CNN)-based computer vision systems. Additionally, there are numerous circumstances where security, privacy, and limitations pertaining to the accessibility of the required equipment make it arduous to validate computer vision systems with real-world datasets. In this paper, we investigated the possibilities of using synthetic datasets, generated from Virtual Environments (VE) for training and validation of CNN models. We present two use cases where the above-mentioned circumstances play a vital role in preparing the datasets and validating the model with large-scale datasets. By developing and leveraging a three-dimensional Digital Twin (DT), we produce large scale datasets for validating social distancing in workspaces;and in the context of semi-autonomous vehicles, we evaluate how a CNN-based object detection model would perform in an Indian road scenario. © 2021 ACM.